Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

(v2.11) Stream ingest rate limiting #5796

Merged
merged 1 commit into from
Sep 7, 2024
Merged

(v2.11) Stream ingest rate limiting #5796

merged 1 commit into from
Sep 7, 2024

Conversation

neilalexander
Copy link
Member

@neilalexander neilalexander commented Aug 16, 2024

Built on top of #5575. Tries to prevent the stream msgs queue from becoming overwhelmed by core NATS publishes. In this case, where a reply subject is known, the sender will receive a 429 "Too Many Requests". Otherwise it's rate-logged.

Two new configuration options are added to the JetStream block: max_buffered_size and max_buffered_msgs. If not configured, defaults are used.

Signed-off-by: Neil Twigg [email protected]

@neilalexander neilalexander force-pushed the neil/jspublish branch 2 times, most recently from 97e36ea to 615ac9d Compare August 30, 2024 11:29
@neilalexander neilalexander changed the base branch from main to neil/ipqsize August 30, 2024 11:29
@neilalexander neilalexander force-pushed the neil/jspublish branch 2 times, most recently from 60a4b62 to 46a3a53 Compare September 2, 2024 15:59
Base automatically changed from neil/ipqsize to main September 4, 2024 20:37
@neilalexander neilalexander force-pushed the neil/jspublish branch 4 times, most recently from 0c70887 to 0ac8594 Compare September 6, 2024 13:21
@neilalexander neilalexander marked this pull request as ready for review September 6, 2024 13:47
@neilalexander neilalexander requested a review from a team as a code owner September 6, 2024 13:47
server/stream.go Outdated Show resolved Hide resolved
server/stream.go Outdated Show resolved Hide resolved
@neilalexander neilalexander force-pushed the neil/jspublish branch 2 times, most recently from b37bd8b to 22a1d66 Compare September 6, 2024 17:22
Copy link
Member

@wallyqs wallyqs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Member

@derekcollison derekcollison left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in general LGTM, a few small comments / questions.

server/opts.go Outdated Show resolved Hide resolved
server/opts.go Outdated Show resolved Hide resolved
server/opts.go Outdated Show resolved Hide resolved
server/opts.go Outdated Show resolved Hide resolved
server/opts.go Outdated Show resolved Hide resolved
@neilalexander neilalexander force-pushed the neil/jspublish branch 3 times, most recently from 1aba888 to eef5abf Compare September 6, 2024 23:41
Copy link
Member

@derekcollison derekcollison left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@derekcollison derekcollison merged commit 7f92c34 into main Sep 7, 2024
5 checks passed
@derekcollison derekcollison deleted the neil/jspublish branch September 7, 2024 18:55
@wallyqs wallyqs changed the title Stream ingest rate limiting (v2.11) Stream ingest rate limiting Oct 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants